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ABSTRACT 

We  consider  the  problem  of  statically  assigning  many  tasks  to  a  (smaller)  system  of  homogeneous 
processors,  where  a  task’s  structure  is  modeled  as  a  branching  process,  and  all  tasks  are  assumed 
to  have  identical  behavior.  We  show  how  the  theory  of  majorization  can  be  used  to  obtain  a  partial 
order  among  possible  task  assignments.  Our  results  show  that  if  the  vector  of  numbers  of  tasks 
assigned  to  each  processor  under  one  mapping  is  majorized  by  that  of  another  mapping,  then  the 
former  mapping  is  better  than  the  latter  with  respect  to  a  large  number  of  objective  functions.  In 
particular,  we  show  how  measurements  of  finishing  time,  resource  utilization,  and  reliability  are  all 
raptured  by  the  theory.  We  also  show  how  the  theory  may  be  applied  to  the  problem  of  partitioning 
a  pool  of  processors  for  distribution  among  parallelizable  tasks. 
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1  Introduction 


F’arallel  processing  lias  emerged  as  an  imiiortant  means  of  achieving  high  computational  perfor¬ 
mance.  As  a  consecpience,  much  research  interest  has  been  sparked  in  the  area  of  efficient  use  of 
jiarallel  computers.  The  jiroblem  of  assigning  tasks  among  processors  to  minimize  processing  time 
has  already  received  considerable  attention  in  the  literature,  e.g.,  [3,  4,  8,  9,  12,  18].  We  consider 
the  problem  of  statically  assigning  tasks  to  processors  when  the  tasks  have  unknown  random  pro- 
ce.ssing  times  and  a  cert.ain  type  of  stochastic  structure.  The  structure  we  examine  embodies  the 
notion  of  one  task  sjiawning  a  set  of  others;  we  examine  static  assignments,  under  the  assumption 
that  all  offspring  of  a  task  are  executed  on  the  same  processor  as  the  task.  Static  assignment  is 
likely  to  be  used  when  a  task’s  state  is  large,  thereby  making  dynamic  assignment  very  costly  in 
terms  of  communication. 

This  pa])er  examines  theoretical  issues  associated  with  compaiing  different  static  mappings  of 
a  set  of  complex  stochastic  tasks.  In  particular,  we  show  how  the  theory  of  inajorization  can  be 
used  to  derive  strong  results  concerning  the  comparison  of  different  mappings.  The  strength  of 
our  contribution  lies  in  our  providing  a  formal  underpinning  to  the  analysis  of  mapping  complex 
stochastic  tasks  and  to  the  optimization  of  a  rich  class  of  objective  functions. 

Previous  work  on  load  balancing  or  task  assignment  [3,  4,  7,  8,  9,  12,  18]  in  parallel  systems 
may  be  loosely  divided  into  three  categories.  The  first  category,  with  deterministic  structure, 
involves  task  structures  and  execution  times  which  are  known  prior  to  assignment.  In  this  case 
[14]  includes  a  study  of  problem  complexity  under  various  constraints  and  heuristic  algorithms  for 
task  scheduling.  A  .second  class  of  load  balancing  formulations,  in  which  task  execution  times  are 
random,  is  characterized  by  (pieueing-theoretic  considerations  [4,  16,  18].  Much  of  this  work  pertains 
to  steady-state  expectations  of  task  delays  with  state-dependent  [4,  18]  and  state-independent  [16] 
assignment  policies.  Our  work  is  closest  to  the  third  category  [7,  8,  9,  13]  which  also  takes  task 
execution  times  to  be  random  but  focuses  on  minimizing  expected  processing  times  for  a  fixed  set 
of  tasks.  As  discussed  in  [9],  the  assum])tion  of  random  execution  times  and  a  given  set  of  tzisks  is 
justified  in  ai)plications  such  as  Monte-('arlo  simulations. 

Our  ai)|)roacli  to  the  problem  differs  from  previous  work  [7,  8,  9,  13]  in  several  ways.  In 
this  paper,  we  do  not  concern  ourselves  with  the  explicit  optimization  of  task  assignment,  but 
rather,  with  the  com])arison  between  different  assignments  over  a  wide  range  of  possible  objective 
functions.  Past  approaches  typically  address  the  question:  given  K  processors  and  m  tasks  with 
random  execution  re<|uirements,  find  the  assignment  of  tasks  to  processors  that  minimizes  the 
expected  maximum  workload  (or  makcspan).  In  this  paper,  we  address  a  related  question:  given 
two  assignments,  when  can  we  say  that  one  is  “better”  than  the  other,  and  for  what  class  of 
objective  functions  can  we  make  this  as.sertion?  Our  results  have  a  simple  general  form.  We  can 
describe  a  mapping  of  |)rol)al)ilistically  homogeneous  tasks  to  processors  by  a  vector  m,  whose 


I 


ith  coin])onent  is  tlio  number  of  tasks  assigned  to  the  ith  processor.  Let  m  and  m'  describe  two 
different  map])ings.  Then  if  m  can  be  bounded  by  m'  using  the  notion  of  majorization  [10]  (written 
m  ^  m'),  then  for  all  objective  functions  /  in  a  class  C  we  may  say  that  the  assignment  described 
by  m  is  better  than  the  assignment  described  by  m' .  The  class  C  is  often  quite  general,  and 
includes  many  commonly  used  objective  functions,  e.g.,  the  expected  maximum  workload.  We  note 
that  an  interest  in  inecpialities  or  siuvhastic  orderings  cki\  be  more  useful  than  merely  searching  for 
optimal  assignments,  because  such  orderings  may  be  derived  in  a  variety  of  cases  where  it  is  too 
ex])ensive  to  search  for  an  o])timal  assignment.  Inequalities  are  also  useful  when  constraints  on  the 
assignment  (e.g.  heterogeneous  memory  capacity  among  processors)  prohibit  one  from  adopting 
an  otherwise  ob  vious  optimal  policy.  We  note  that  stochastic  orderings  are  of  independent  interest 
[l.'i]  and  also,  in  some  of  the  cases  we  consider  the  optimal  strategy  is  apparent  from  the  derived 
ordering. 

Our  interest  in  obtaining  stochastic  orderings  also  stems  from  the  observation  that  they  are  often 
the  only  results  availalile  hrr  small  numbers  of  random  variables  and  a  wide  variety  of  distributions, 
('onsider  the  fact  that  in  [S,  0]  the  results  are  asymptotic  in  at  least  one  variable  n  or  K.  In  fact, 
in  [n],  the  results  are  only  asymiitotically  correct  in  both  the  number  of  tasks  n  and  the  number 
of  iirocessors  A'.  These  approaches  are  based  on  the  use  of  the  central  limit  theorem  [8]  and  large 
deviation  theory  [9],  which  are  among  the  few  limit  results  available  that  hold  for  a  variety  of 
distributions.  In  contrast,  our  approach  is  concerned  with  finite  (and  possibly  small)  n  and  K  and 
we  make  use  of  the  theory  of  stochastic  majorization  [10].  Thus,  while  some  of  our  results  are  not 
as  strong  (in  terms  of  optimality)  as  those  obtained  from  fundamental  limit  theorems,  the  accuracy 
of  our  results  does  not  de|)end  on  the  number  of  tasks  or  processors. 

We  now  discuss  other  specific  differences  between  our  work  and  past  efforts.  Our  structural 
model  of  a  single  task  is  that  of  a  branching  process:  a  completing  process  spawns  a  random  number 
of  subprocesses.  This  tyjie  of  behavior  apjiears  in  diverse  applications  such  as  Branch-and-Bound 
searching  algorithms  [2]  where  the  branching  structure  is  obvious,  and  dynamic  regridding  algo¬ 
rithms  in  numerical  computations  [1]  where  sections  of  coarse  grid  serve  as  “processes”  which  give 
rise  to  “subprocesses”  associated  with  finer  grids.  Furthermore,  our  results  permit  the  analysis  of 
much  more  complex  objective  functions  than  have  typically  been  studied  for  stochastic  task  models. 
Our  model  differs  significantly  from  those  in  [H,  9,  12].  The  tasks  in  [9]  were  taken  to  be  individ¬ 
ual  independent  and  identically  distributed  (i.i.d.)  samples  drawn  from  a  common  distribution, 
and  synchronization  behavior  is  that  of  periodic  global  .synchronization.  In  [8]  a  complex  task  is 
comiirised  of  a  fixed  number  of  tasks  with  random  i.i.d.  execution  times.  However,  the  analyses 
in  both  [9]  and  [S]  are  concerned  with  overheads  (e.g.  .synchronization  and  communication  costs) 
that  our  model  does  not  include.  In  .some  ways  the  present  work  resembles  earlier  results  obtained 
under  the  assumption  that  the  workload  a.s.signed  to  a  processor  causes  the  processor  to  behave 
as  a  Markov  chain  [!.{].  Like  this  earlier  work,  our  new  results  show  how  the  quality  of  a  static 
assignment  persists  across  numerous  stochastic  transformations  of  the  workload.  The  model  we 
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study  in  the  pioseul  i)ai)('r  is  a  distinct  improvement  over  that  in  [13],  as  the  stochastic  behavior 
of  a  processor  is  now  explicitly  de|)endent  on  the  volume  of  workload  it  contains. 

Other  related  research  has  been  directed  at  computing  the  expected  completion  time  for  a 
ahifjle  complex  task  with  a  possibly  random  acyclic  structure  [6,  17].  Another  related  publication 
[11]  studies  the  problem  of  scheduling  sub-tasks  of  a  single  task,  where  the  sub-tasks  form  a  tree. 
Lastly,  an  analytic  study  of  load-balancing  statistically  homogeneous  workload  on  a  hypercube  is 
])resented  in  [7],  where  the  mean  and  variance  of  the  difference  between  the  load  on  a  processor 
and  the  average  load  are  derived.  While  past  research  has  been  concerned  exclusively  with  a  single 
task  or  a  given  set  of  tasks,  we  also  consider  the  joint  assignment  of  multiple  classes  of  tasks,  where 
tasks  in  different  clas.ses  have  different  probabilistic  behaviors. 

Our  work  is  ba.sed  on  results  from  the  study  of  stochastic  majorization.  The  fundamental  theory 
of  majorization  originates  in  the  economic  study  of  income  distribution — a  sort  of  “load”  balancing. 
We  believe  majorization  finds  a  natural  ai)plication  in  the  area  of  mapping  parallel  workload,  and 
that  one  of  our  contributions  is  to  demonstrate  uses  of  this  powerful  theory  in  parallel  processing. 
In  this  respect  our  work  is  similar  to  that  in  [3,  19].  In  [.3]  the  focus  is  on  a  new  stochastic  ordering 
based  on  the  class  of  symmetric,  convex  and  L-subadditive  functions  with  applications  to  routing 
and  designing  i)rocessor  speeds.  The  load  balancing  emphasis  in  [3]  is  on  scheduling  structurally 
simple  tasks  from  a  (pieue.  Majorization  in  steady-state  queue  lengths  of  open  queueing  networks  is 
studied  in  [10],  in  which  orderings  are  j)arameterized  by  queue  utilizations.  In  contrast,  we  use  the 
established  orderings  in  [10]  to  obtain  inequalities  among  all  generations  of  complex  tasks  under 
different  static  map|)ings  of  the  initial  tasks. 

The  rest  of  this  pajter  is  organized  as  follows.  In  the  next  section,  we  define  basic  notation 
and  ])resent  our  workload  model;  also,  we  discuss  the  different  stochastic  orderings  to  be  used 
throughout  the  paper.  Section  §3  contains  the  fundamental  orderings  for  workloads.  Section  §4 
discusses  various  objective  functions  of  interest  in  parallel  systems  and  Section  §5  applies  the  theory 
to  the  problem  of  ])artitiouing  a  pool  of  processors  among  a  set  of  parallelizable  tasks.  Section  §6 
summarizes  our  work. 

2  Preliminaries 

We  now  introduce  our  model  of  computation,  important  definitions  and  known  results,  and  a 
rationale  for  using  majorization  to  study  the  assignment  problem. 

2.1  Workload  and  System  Model 

We  model  the  workload  produced  by  a  single  task  as  a  branching  process  [1.5,  pp.  116-117],  as 
follows.  The  task  begins  with  a  single  inoik  unit  (WU)  of  computation.  The  WU  is  executed;  upon 
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its  completion  a  random  number  of  other  WUs  are  created,  and  placed  in  the  task’s  work  list. 
The  initial  VVU  can  thus  be  thought  of  as  containing  the  “seeds”  for  a  number  of  additional  WUs, 
possibly  zero,  each  of  which  similarly  contain  the  seeds  for  additional  WUs,  and  so  on.  One  of 
the  first  generation  WUs  may  then  be  executed,  and  its  children  (which  are  2"'^  generation  WUs) 
spawned  and  i)laced  in  the  task’s  work  list.  The  number  of  children  a  WU  spawns  is  assumed  to  be 
random,  chosen  from  a  |)robability  distribution  known  as  the  branching  distribution.  The  process 
is  repeated  until  the  task’s  work  list  is  empty.  The  task  workload  is  comprised  of  all  computation 
related  to  all  WUs  ultimately  descended  from  the  initial  task  WU. 

We  assume  that  the  order  of  WU  execution  in  no  way  affects  the  spawning  of  children:  a  WU  in 
the  work  list  is  destined  to  spawn  some  j  children,  regardless  of  the  length  of  time  it  spends  in  the 
list.  This  is  easily  understood  if  one  views  the  WU  generation  as  reflecting  some  intrinsic  structural 
j)roperty  of  the  problem,  e.g.,  the  branching  of  a  search  tree.  Because  of  this  independence,  every 
WU  belongs  to  .some  "generation”  which  is  independent  of  execution  order.  The  initial  WU  is  in 
generation  0;  all  children  spawned  by  a  generation  1  WU  are  in  generation  2,  and  so  on. 

We  assume  that  a  given  WU  may  be  executed  with  the  same  constant  cost  on  any  one  of  K 
homogeneous  i)rocessors,  and  that  every  WU  is  executed  on  the  same  processor  as  is  its  parent. 
Therefore,  we  map  all  computation  associated  with  a  task  when  we  map  the  task’s  initial  WU. 

Consider  the  evolution  of  an  initial  task  WU.  Let  Ng  denote  the  number  of  WUs  in  its  9-th 
generation.  The  size  of  the  (/-th  generation  is  given  as 

/v,_, 

'v,  =  E  Z;...  (>) 

j=i 

where  No  =  1  and  where  Z,  ,,  is  the  number  of  WUs  generated  by  the  j-th  WU  in  the  (q  —  l)-th 
generation.  We  assume  that  {Zj.,,,  1  <  q  <  h'}^j  is  a  sequence  of  independent  and  identically 
distributed  (i.i.d.)  r.andom  variables  (r.v.’s).  The  following  notation  will  be  employed: 

•  l\  the  number  of  processors. 

•  n  -  the  number  of  initial  task  WUs. 

•  m  an  integer  as.signment  vector  who.se  component  nii  gives  the  number  of  WUs  assigned 
to  the  i)roce.ssor. 

•  Ng  -  the  size  of  generation  r/,  descended  from  a  single  initial  WU  (when  the  branching 
distribution  is  understood).  For  any  subset  A  C  N,  Sa  >s  the  sum  of  aU  sizes  of  generations 
i  e  A: 

Sa  =  E 
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•  -  the  convolution  of  a  pro1)ability  mass  function  /.  If  X  is  a  random  variable,  we  will 
also  use  A'*-'*  to  denote  a  sum  of  j  independent  instances  of  X . 

•  Wq{Tn)  -  the  random  vector  of  generation  q  WUs  resulting  from  assignment  vector  m: 

We  denote  the  component  of  Wq{m)  by  {Wq{m))..  The  notation  is  extended  to  arbitrary 
subsets  A  C  SV  hy 

The  theory  we  develo])  permits  us  to  compare  different  mappings  under  a  variety  of  objective 
functions  <!>  :  ff/'  — ♦  R.  Our  results  focus  on  comparing  values  of  E[<i>[W q{m))]  by  deriving 
conditions  for  inequalities  involving  initial  task  assignments  m.  Most  of  these  are  of  the  following 
form:  given  two  assignments  m  and  m'  where  m  <  m'  (see  Definition  2.1),  then  E[4){W A{m))]  < 
E  [(l){W Aifn'))]  for  all  subsets  A  C  IV,  when  the  expectations  exist. 

Applicable  functions  0  include  any  symmetric  convex  function;  the  maximum  operator,  all 
powers  of  the  maximum,  the  sum  operator,  and  the  product  operator  are  of  particular  interest. 
Thus  a  single  comparison  between  the  assignment  vectors  m  and  m'  vectors  can  yield  a  wealth  of 
information  about  the  comparative  behaviors  of  complex  stochastic  tasks  under  the  two  mappings. 

Our  results  are  ap|)licable  to  two  different  types  of  processor  synchronization.  We  study  gen¬ 
erational  synchronization  (Cl.S)  where  processors  engage  in  a  barrier  synchronization  between  each 
WU  generation.  A  processor  executes  all  WUs  of  a  given  generation,  say  q,  then  synchronizes  at 
the  barrier.  It  is  not  released  until  all  processors  have  executed  all  their  generation  q  WUs  and 
reached  the  barrier.  The  ])rocess  re])eats  for  subsequent  generations.  This  type  of  synchronization 
is  appropriate  when  the  computation  for  a  generation  q  in  one  task  may  depend  on  results  computed 
by  a  generation  q  —  I  WU  in  another  task.  We  also  study  termination  synchronization  (TS),  where 
a  processor  engages  in  a  barrier  synchronization  only  after  the  work  lists  of  all  its  initial  tasks  are 
empty.  This  is  appropriate  when  the  tasks  are  independent  of  each  other,  and  the  synchronization 
serves  only  to  aggregate  the  final  results  of  their  respective  computations. 

Not  sur])risingly,  the  optimal  way  of  assigning  n  tasks  to  K  processors  is  usually  to  assign  nj P 
to  each.  In  the  face  of  the  obvious  one  may  well  ask  why  we  study  partial  orderings.  Primarily,  the 
theory  ])roves  the  optimality  with  respect  to  a  large  number  of  objective,  functions,  thereby  lending 
theoretical  support  to  intuition.  Secondly,  the  theory  works  even  in  the  presence  of  constraints 
that  disallow  the  uniform  assignment,  and  complicate  one’s  intuition  concerning  optimality.  For 
example,  memory  constraints  may  exist  that  forbid  one  or  more  processors  from  being  assigned  more 
than  nf  P  tasks.  The  theory  identifies  the  optimal  assignment  under  heterogeneous  constraints. 

We  will  also  apply  these  concepts  to  the  issue  of  partitioning  a  pool  of  processors  among  a 
set  of  complex  parallelizable  tasks.  Here  we’ll  take  K  to  the  be  number  of  paraUelizable  tasks. 


and  use  m  to  describe  the  number  of  processors  assigned  to  each.  Constraints  on  feasible  m 
are  easily  envisaged,  as  the  assignment  may  need  to  consider  “natural”  partition  sizes  that  arise 
from  communication  to])ology,  or  system  usage  at  the  time  of  the  assignment.  So  again,  while  the 
optimal  solution  to  the  constraint-free  version  of  the  problem  may  be  apparent,  the  theory  provides 
a  means  of  com|)aring  feasible  solutions. 


2.2  Stochastic  Ordering  and  Majorization 

We  now  introduce  the  majorization  partial  ordering  -<  using  notation  largely  taken  from  [10]. 
Definition  2.1  (majorization)  A  victor  x  in  majorized  by  vector  y,  writtcri  as  x  -<y,  iff 

I-  ELi  ^-[i]  <  ELi  y[i]>  -  1, •  •  •, »  -  1, 

E"=i  •'■■[.]  =  E’=i  yw- 

luliere  the  notation  is  taken  to  be  the  i-tli  largest  element  of  x. 

Definition  2.2  (Schur-convex  function)  A  function  <f>  •.  R"'  ^  R  is  said  to  be  Schur-convex  if 
X  <  y  in  R'^  implies  (hix)  <  (l>{y)  in  R. 


Examples  of  Schur-convex  functions  include  <l>ix)  =  maxx,-  and  <i>ix)  =  Ey(^«)'  ^  convex  g  :  R 
R. 

Let  t’o  he  the  class  of  increasing  functions  from  i?"  onto  R.  The  well-known  stochastic  ordering 
between  random  variables  [L')]  is  defined  as  follows.  For  random  vectors  X  and  Y  with  distribution 
functions  F  and  (I  respectively. 


X  <,t  Y 


<l>{x)dG{x) 


V(/>  e  Co 


such  that  the  integrals  are  well  defined.  Majorization  over  deterministic  quantities  is  extended  to 
random  variables  in  like  manner  by  using  an  appropriate  class  of  functions: 


C]  =  {.sc;r}  =  {/  :  R.  \  f  Schur-convex  }, 

Cl  =  {ctt.s}  =  {/  ;  —*  R.  \  f  convex  and  symmetric  }. 


These  define  respectively  the  Schur-convex  partial  ordering,  denoted  by  and  the  convex  sym¬ 
metric  partial  ordering,  denoted  by  -<cas  (the  notation  <E\  and  is  used  in  [10]).  Thus, 
X  Y  iff 

Hx)dF(x)  <  (l>(x)da{x)  V<^  e  Cl 
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and  X  y  iff 


j^^^<t>{x)dF{x)<  j^^4>{x)dG{x)  ^4,eCi. 

Note  that  C2  C  C\  and  thus,  -<scx  is  a  stronger  ordering  than  ■<cas- 

Stochastic  orderings  based  on  likelihood  ratio  play  an  especiaUy  important  role  in  this  paper, 
('onsider  non-negative  integer  valued  r.v.’s  X  and  Y  with  probability  mass  functions  /  and  g. 


Definition  2.3  (likelihood  ratio)  X  is  defined  to  be  smaller  than  Y  in  likelihood  latio,  written 

as  X  <ir  Y,  iff 


/(»0  ^  fiffl 

g(m)  -  g{n)' 


0  <n  <  7/1,  n,jn  G 


Another  important  property  for  a  probability  distribution  is  known  as  increasing  likelihood  ratio. 


Definition  2.4  (ILR)  The  non-negatiw.  integer  valued  r.v.  X  i.<;  said  to /mve  increasing  likelihood 
ratio  (ILR)  (and  its  probability  mass  function  f  is  said  to  be  ILR)  iff 

<1  +  A'  <ir  C.2  +  X,  whenever  0  <  ci  <  c 2. 


Next  we  define  another  class  of  probability  mass  functions,  those  which  have  increasing  likelihood 
latio  under  convolution. 

Definition  2.5  (ILRC)  Let  f  be  a  probability  mass  function  defined  on  iV.  /  is  said  to  have 
increasing  likelihood  under  convolution  (ILRC)  iff  f^'^  </r  whenever  i  <  j. 

ILR  distributions  are  known  to  be  closed  under  convolution,  even  when  the  number  of  times  con¬ 
volution  is  applied  is  random  (provided  the  distribution  of  this  number  is  also  ILR)  [10]. 

Lemma  2.1  Let  f  be  an  ILR  probability  mass  function.  Then 

•  /  is  ILRC. 

•  For  any  fixed  integer  k  >  0,  is  ILR. 

•  Let  N  be  an  ILR  jmsitive  integer-valued  random  variable.  Then  f^^^  is  ILR. 

Using  these  facts  it  is  straightforward  to  prove  the  following. 

Lemma  2.2  Let  f  be  an  ILR  probability  ma.ss  function.  Then 
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•  If  f  is  till  bvaiu  hiny  distribution  for  a  task,  then  for  all  gcneiations  q,  Nq  is  HR. 

•  For  any  snb.'ut  AC  A’,  if  S  =  XI, N,  has  finite  mean,  then  is  ILRC. 

Proof:  Tli('  proof  of  the  first  rlahn  is  a  simple  induction  on  q  that  uses  closure  of  the  ILR  property 

under  random  ILR  mixtures;  the  proof  of  the  second  rewrites  as  +  c,,  where  q  is  the  least 
element  of  ,4,  an  cl  r,  <  Cj  .almost  surely  whenever  i  <  j.  The  result  follows  from  Definition  2.4  and 
the  f.act  th.at  .V,,  is  ILR.  ■ 

.4s  we  will  see,  the  .assumption  of  an  ILR  branching  distribution  often  yields  orderings. 
The  ILR  condition  is  true  of  the  discrete  Uniform,  F'oisson,  (leometric  and  Binomial  distributions, 
showing  that  our  results  ap|)ly  when  the  Pranching  assumes  some  well-known  distributions. 

.Next  we  show  how  these  stoch.astic  orderings  may  be  used  to  develop  stochastic  majorizations 
between  different  st.atic  m.a|)pings. 

3  Branching  and  Stochastic  Majorization 

In  this  section  we  est.ablish  conditions  under  which  either  -<on»  or  -Kgcx  orderings  can  be  established 
between  ••workIoa<r’  vectors  under  different  mappings.  The  notion  of  workload  will  be  seen  to 
be  (juite  general.  Throughout  this  section  it  is  important  to  remember  that  the  results  relate  to 
intrinsic  projierties  of  branching  beh.avior.  and  do  not  depend  on  assumptions  about  execution 
behavior,  e.g.,  synchronization. 

Our  results  for  the  ordering  is  b.xsed  on  the  following  theorem  which  is  an  application  of 
Theorem  ;{..1.2  in  [10].  The  correspondence  between  our  form  and  the  original  is  pointed  out  in  the 
Apj)endix. 

Theorem  3.1  Let  f  be  an  ILRi'  probability  mass  function,  let  m  —  {in\, . . .  be  a  vector  of 

nonneyative  integers,  and  for  each  j  -  I , . . . ,  K  let  be  a  r.v.  with  distribution  .Suppose 

this  si  t  of  r.r.s  is  independent,  and  let  <l> :  ^  H  be  a  Schur-convex  function.  Then 

7(m)  =  ^  [0(4r  A' <”*^>)] 

is  a  Schur-convex  function  of  m. 

Using  Theorem  .'f.l  we  obt.ain  our  basic  ^sa  ordering  results. 

Theorem  3.2  ('onsidcr  a  si  t  of  n  tasks,  with  common  ILR  branching  distribution  f,  and  let  m 
and  m'  be  twn  mnpping  victors  .such  that  m  -<  m' .  Then 


H 


•  For  all  (j(iu  ratiiivs  q,  W,i{m)  Xs.j-  W,,(Tn'). 

•  For  any  s\bs(  t  A  C  iV  such  that  has  finite  mean, 

W Aim)  -<„cx  W A{m'). 

Proof.  L**nima  2.2  .slmws  that  thr  distributions  of  Ng  and  .S"^  are  each  ILRC;  the  result  follows 
from  the  definitions  of  Wglm)  and  W.j(m),  and  Theorem  d.l.  I 

Observe  that  the  statement  of  Theorem  H.l  applies  more  generally  to  the  notion  of  a  random 
"reward"  as.sociated  with  each  initial  \Vl'.  It  states  that  if  each  initial  WU  earns  a  random  ILRC 
reward,  and  if  the  reward  to  a  processor  is  the  sum  of  the  rewards  earned  by  its  (independent) 
VVl^'s,  then  a  stochastic  majorization  on  the  rewards  follows  from  a  deterministic  majorization  of 
the  initial  VVT's. 

Our  results  seem  to  re<|nire  the  assumption  of  ILR  or  ILRC  branching  distributions. 

However.  I)y  i  (iiistrainiiig  our  attention  to  symmetric  convex  functions  we  are  able  to  obtain  ■<cas 
orderings  for  completely  general  branching  distrilmlions.  The  details,  which  are  numerous,  are 
develo|)ed  in  the  .Appi'iidix.  The  ,,  counterpart  to  Theorem  3.2  is 

Theorem  3.3  Consiih  r  a  sit  nj  n  tasks,  with  eommon  nonnegative  branehing  distribution  /.  and 
lit  m  and  m'  Im  tiro  mapping  vietors  .ss.  that  m  -<  tn'.  Then 

•  For  all  (p  ni  rations  q.  W.,(  m) 

•  For  any  subsi  t  .1  C  A  such  that  .S' has  finitt  mean. 

W  aim)  <,.„s  W A(m'). 


3.1  Heterogenous  Constraints 

The  A- vector  m„pt  =  {n/l\,n/l\ _ ,n./l\)  is  majorized  by  any  other  vector  whose  components 

are  nonnegative  and  sum  to  n.  Applied  to  the  a.ssignment  problem,  this  shows  that  the  obvious  way 
to  balance  worklo.ad  is  indeed  the  best,  even  for  complex  stochastic  tasks.  Optimality  is  less  clear, 
however,  if  the  obvious  assignment  is  prohibited  by  constraints.  For  each  processor  i  let  C'i  be  an 
upper  bound  on  the  numl)er  of  VVCs  the  |)rores.sor  may  be  given.  .Such  constraints  might  arise,  for 
instance,  if  the  proces.sors  have  different  memory  capacities.  The  obvious  mapping  is  prohibited  if 
any  <",  <  n  j K .  Majorization  provides  a  way  to  identify  the  best  assignment  of  complex  stochastic 
tasks  even  in  the  face  of  such  constraints. 
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( 'onsidpr  any  foasil)l('  vector  y  =  (^i , . . . ,  j//v),  Vi  <  Ct  for  i  =  Suppose  there  exist  i 

and  j  siicli  that  i/j  >  ij,  +  1.  and  y,  +  I  <  (Construct  a  new  vector  x  from  y  by  transfering  one 
unit  from  to  i.e.,  .r,  =  i/j  -  1,  x,  =  y,  +  1,  xi^  =  yt^  for  all  k  7^  i,j.  It  is  shown  in  [10]  (5.D)  that 
X  <  y.  rids  observation  "ives  a  rule  by  which  we  can  iteratively  improve  a  feasible  solution,  until 
no  further  improvement  is  possible.  We  say  a  vector  x  resulting  from  this  processed  is  balanced. 

Without  loss  of  f!;<>n<'rality  assume  that  ('x  <  ('i  <  ■••  <  ('k-  It  is  apparent  that  x  is  balanced 
if  and  only  if  whenever  x,  >  x,  +  1,  then  x,  =  A  characterization  of  balanced  vectors  then 
is  that  there  is  some  index  j  such  that  x,  =  C,  for  i  =  l,...,j,  and  for  all  l,m  >  j  we  have 
i-^'t  -  'i/il  <  1-  Furthermore,  if  x  and  y  are  both  balanced,  then  this  index  j  is  the  same  for  both 
of  them.  It  follows  then  that  x  -<  y  and  y  x  x,  which  shows  the  essential  uniqueness  of  balanced 
vectors.  Halanc('d  vectors  are  thus  optimal  under  heterogenous  constraints. 

.\  simple  algorithm  will  construct  a  balanced  assignment.  Assume  the  processors  are 

ordered  by  incrt'asing  constraint  value,  and  initially  set  x,  =  0,  i  =  1,2,...,  A'.  We  loop  repeatedly 
over  indices  1  to  A'.  Facli  pass  thr<jugh  the  loop  we  increment  :r,  once,  provided  jr,  <  C',.  This 
('ssentially  assigns  one  unit  to  the  processor.  We  repeat  the  loop  until  all  n  units  are  assigned. 

The  main  results  of  these  .section  show  that  stochastic  branching  preserves  stochastic  majoriza- 
tion  for  additive  reward  .systems.  As  we  have  seen,  useful  reward  systems  are  derived  from  the 
generation  sizes.  The  section  to  follow  illustrates  how  these  results  can  be  fruitfully  appUed  to 
various  objective  functions. 


4  Objective  Functions 

We  will  now  establish  that  a  number  of  interesting  objective  functions  are  either  Schur-convex  or 
convex  symmetric  functions  of  some  notion  of  workload.  These  objective  functions  include  finishing 
time  under  diJerent  synchronization  schemes,  the  space-time  product,  and  overall  reliability.  This 
diversity  of  application  demonstrates  the  utility  of  the  theory. 

4.1  Finishing  Time 

OiH'  use  of  majorization  is  to  show  that  whenever  m  -<  m',  the  computation’s  expected  finishing 
time  under  m  is  better  than  that  under  m' .  This  can  be  established  using  different  models  of 
execution.  For  example,  one  easily  envisions  a  computation  where  the  tasks  must  synchronize 
globally  after  every  generation,  i.e.,  (IS  "mchi-onization.  This  is  typical  of  tasks  associated  with 
numerical  computations.  If  the  WUs  each  have  unit  execution  time,  then  max*;{(Wg(m))^.}  time 
is  re(|uired  under  ma|)|)ing  m  to  execute  all  generation  q  WUs.  Ng  can  be  viewed  as  a  random 
reward  associated  with  an  initial  WU,  thus  Theorem  3.3  tells  us  that  Wg(m)  Scaa  Wg{m').  The 
max  operator  is  convex  and  symmetric,  whence  E  [maXA,{(VF,(m))^}]  <  E[  iax;L.{(iy,(m'))^,}] . 
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This  same  ipsiilt  holds  true  if  the  VVU  execution  times  are  random,  and  i.i.d.  .  Since  the  time 
between  each  synchronization  is  no  larger  under  m  than  than  under  m',  it  follows  that  the  overall 
finishing  time  is  no  larger. 

Similar  results  are  obtained  under  TS  synchronization,  where  processors  synchronize  only  at 
termination.  The  reward  for  an  initial  WU  can  be  defined  to  be  Su,  the  total  size  of  the  branching 
tree  rooted  in  that  WU.  When  the  mean  of  the  branching  distribution  is  strictly  less  than  one,  then 
<  oo-  III  this  case,  whenever  m  -<  m' ,  the  expected  maximum  processor  reward  under  m 
is  no  larger  than  under  m'.  Even  when  the  branching  distribution’s  mean  is  greater  than  or  equal 
to  one  (but  is  finite)  we  can  always  assert  that  the  time  to  execute  all  generations  up  through  q  is 
no  greater  under  m  than  it  is  under  m',  by  defining  the  reward  for  an  initial  WU  to  be  the  sum  of 
the  sizes  of  generations  through  q.  Any  symmetric  convex  function  of  the  processor  rewards — such 
as  the  maximum  ])rocessor  reward — yields  an  -<cas  ordering. 

Another  metric  of  interest  is  the  variation  in  the  time  to  synchronize.  The  sample  variance, 
defined  below,  is  also  .symmetric  and  convex. 

SampleVar{x)  = 


where  x  =  Thus, 

S ainplcV ar{W g{m))  -<cas  SanipleVar{W 


for  any  generation  q,  and 

SanipleV ar{W A{m))  -<cas  SanipleVar(W g{m')) 

for  any  A  C  N  such  that  Sa  lias  finite  mean.  When  the  branching  distribution  is  ILR,  a  similar 
result  holds  true  for  the  sample  standard  deviation  (square  root  of  variance)  of  time  between 
synchronizations,  becau.se  the  standard  deviation  is  Schur-convex  ([10],  pp.  71). 


4.2  Functions  of  Queue  Length 

When  a  WU  rom])letes  its  execution  it  generates  its  children  and  places  them  on  the  processor’s  work 
list.  Following  this,  another  WU  is  selected  to  be  executed.  There  is  thus  a  storage  cost  associated 
with  executing  complex  tasks;  more  generally,  we  show  here  how  stochastic  majorization  can  be 
applied  to  objective  functions  based  on  measuring  queue  lengths  at  every  time  step.  A  simple 
example  of  this  is  the  computation’s  total  .space-time  product,  defined  as  follows.  Let  Q{t)  be  the 
vector  enumerating  the  number  of  WUs  enqueued  at  each  processor  at  time  t,  and  let  T  be  the 
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computation's  tPi  niinatioii  time.  Then  the  total  space-time  product  is  (Q(O)a.- 

idea  can  be  generalized — let  .s(j)  quantify  the  cost  of  holding  j  'WUs  in  queue  for  one  unit  of  time. 
Then  the  total  space-time  cost  with  respect  to  .s  is  "‘*((Q(0)a,)-  "'ll!  show  that  if  .s 

is  increasing  conve.x  with  .s(0)  =  0,  and  if  m  rn',  then  under  TS  synchronization  the  expected 
space- time  cost  with  respect  to  .s  is  no  worse  under  ■m  than  it  is  under  m'.  This  result  is  also 
demonstrated  for  (l.S  synchronization  when  the  branching  distribution  is  ILR. 

Under  the  model  assumi)tions  we  have  made,  the  probabilistic  behavior  of  a  processor’s  queue  is 
completely  indejrendent  of  the  queueing  discipline  used.  We  will  assume  that  the  queueing  discipline 
is  Smallest-Cleneration-First  (SCF):  whenever  a  processor  selects  a  WU  for  execution  from  its  work 
list,  it  chooses  one  with  least  generation  index.  For  simplicity,  we  also  assume  that  the  execution 
of  a  WU  takes  unit  time. 

The  space-time  function  .s(^-)  =  A,'  gives  rise  to  the  usual  space-time  product,  but  other  space- 
time  cost  functions  are  also  intuitive.  For  example,  one  might  have  to  store  WU  states  on  disk 
whenever  the  (pieue  length  exceeds  a  threshold  L;  furthermore,  once  L  is  exceeded  the  cost  might 
be  superline.ar,  owing  to  fragmentation  costs.  A  candidate  cost  function  would  be 


.s(A-)  = 


0  if  k  <  L 

(L  -  A:)'+'  if  k>  L 


where  t  >  0.  The  general  assumptions  that  a  space-time  cost  function  be  convex,  increasing,  and 
zero  for  (Uiipty  cpieue  lists  seem  to  us  quite  natural. 

Our  treatment  of  space-time  costs  under  TS  synchronization  hinges  on  the  following  observa¬ 
tion:  if  |)rocpsst.r  k  has  exactly  (VUq(m))^.  WU  units  in  generation  q,  then  under  the  SGF  queueing 
discipline  at  some  point  in  time  the  processor's  queue  will  have  exactly  {Wg{m))f,  WUs.  In  partic¬ 
ular,  at  the  instant  where  the  first  WU  of  generation  q  is  about  to  be  executed,  the  queue  consists 
entirely  of  generation  q  WUs,  and  contains  all  of  them.  We  will  show  that  the  contribution  to  the 
expected  space  cost  made  by  processor  k  while  processing  generation  q  WUs  (under  SGF  schedul¬ 
ing)  is  an  increasing  convex  function  of  (W,(m))^.,  and  use  this  fact  to  find  a  raajorization  on  the 
vector  of  ex|)ected  contributions  made  by  all  processors  while  processing  generation  q  WUs.  This, 
in  turn,  will  show  that  the  total  expected  space-time  cost  under  m  is  no  worse  than  under  m', 
when  the  expectations  exist.  This  is  a  -<i.„s  result,  applicable  for  any  branching  distribution. 

Suppose  (W,j{m))f.  =  r.  The  proce.ssing  of  the  WU  in  generation  q  {i  =  l,...,r)  produces 
a  random  number  of  WU  units,  who  join  the  processor’s  queue.  The  queue  length  at  the 
instant  the  WU  begins  execution  is  r  —  (t  —  1)  -F  there  were  r  work  units  in  queue 

at  the  point  the  first  generation  q  WU  was  executed,  i  —  1  of  them  have  been  executed,  and  each 
one  |)roduced  a  random  number  of  generation  9+1  WUs.  Therefore,  the  conditional  expected 
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space-time  cost  suffered  during  the  processing  of  this  WU  is 


i-l 


(2) 


(j)  is  convex  in  r,  because  for  any  convex  7  and  random  variable  Z,  the  expectation  E\'){a  -t-  Z)] 
is  convex  in  a  (assuming  the  expectation  exists).  The  expected  space-time  cost  of  processing  all  r 
members  of  generation  q  on  i)rocessor  k  is 


r 

t=i 

Finally,  we  claim  that  f 's(r)  is  a  convex  function  of  r.  To  demonstrate  this  it  suffices  to  show  that 
C's(r-t-2)-|-Ci(7')  >  2Cs(r-|-l)  for  all  r.  Since  <jf>  is  convex  in  r  we  have  r-|-2)-t-<^(j,  r)  >  2(^(j,r-t-l) 
for  all  j  =  1, . .  .r.  This  observation  reduces  the  problem  to  a  demonstration  that 

<l){r  +  2,  r  -f  2)  -|-  </)(r  +  l,r-|-  2)  >  2(t>{r  +  l,r  +  1). 

The  fact  that  s{r)  is  increasing  establishes  that  both  <^(r  -I-  2,r  +  2)  and  <^(r  +  l,r  -|-  2)  dominate 
(t>{r  +  1,  r  -b  1),  thereby  proving  the  convexity  of  Cs(r). 

The  function  7^(ri , . . . ,  r^)  =  is  symmetric  and  convex  on  7V^,  because  whenever 

g  is  convex  on  R  then  h(x)  =  i®  convex  on  .  Observe  that  Ts{Wq{m))  is  the  random 

space-time  cost  with  respect  to  s  and  generation  q  resulting  from  assignment  vector  m.  We  have 
proven  the  following  result. 

Proposition  4.1  Let  s  be  an  increasing  corwex  function  with  s(0)  =  0  and  suppose  the  space  cost 
of  holding  k  Wf/s  in  one  processor’s  queue  for  one  time  unit  is  s{k).  Define 

K 

1=1 

to  measuiT  the.  space-time,  cost  suffered  while  executing  generation  q,  under  the  assignment  given 
by  m.  Then  whenever  m  m' , 

.  E[T,{W,{rn))]<E[Ts{W,{rn'))]forq  =  Q,\,.... 

•  The  expected  total  space-time  cost  using  TS  synchronization  is  no  worse  under  m  than  under 
m  : 

00  00 

E[Y,T,{W,{m))\<E[Y,r,{W,  (m'))]  whenever  the  expectation  exists. 

ij=0  17=0 
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The  analysis  of  space  time  costs  under  CS  synchronization  requires  more  work,  and  the  as¬ 
sumption  of  an  ILR  branching  distribution.  Suppose  that  (Wg(m))^.  =  r^.,  for  k  =  The 

space-time  cost  to  processor  k  during  the  interval  of  time  when  generation  q  WUs  are  executed  has 
two  com|)onents.  We  h.ave  already  seen  the  first:  C{rf;) — the  cost  accumulated  over  the  period  of 
length  7'*;  while  generation  q  WUs  are  executed.  The  second  component  is  the  space-time  cost  suf¬ 
fered  waiting  for  the  most  heavily  loaded  processor  to  finish.  If  processor  k  generates  x  generation 
q+  1  WUs,  then  the  space-time  cost  it  suffers  waiting  at  the  barrier  is  (max,{r,}  — rfc)fl(a;).  Recalling 
the  definition  of  0  (equation  (2))  we  may  write  the  expected  total  space-time  cost  of  processing 
generation  q  WUs  under  GS  synchronization  (conditioned  on  (W,(m))^.  =  r^,  for  k  =  1, . . .,  K)  as 

t;(7'i,...,c/v)  =  ( 

t-i  \ 

Observe  that  (/^(r't.  -f  1,7>)  is  £'[.s( where  X  is  the  branching  random  variable  .  t/  is  Schur- 
convex  on  ,  a  fact  we  show  using  the  following  characterization  of  Schur-convex  functions  on 
(;i.A.2.b  in  [10]). 

A  function  o  on  is  Schur-convex  if  and  only  if  «  is  symmetric  and 
rt(7'i ,  <  -  7'i ,  r;}, . . . ,  rf{}  is  increasing  in  ri  >  t/2 
for  each  fixed  t,  r-.j, . . . ,  7Vc- 


r/c 

E 

i=l 


</4i,n)  +  (max{7j}  -  rfc)0(ri  -|-  l,rt) 

J  j 


Fix  r.}, . . . ,  r/v,  and  consider  ri  >  r-2.  If  the  difference  6^(r]  -f  1,  r2  -  1, . . .,  r/^-)  -  ^(n ,  r2, . . ., r^-)  is 
always  nonnegative,  then  the  condition  above  tells  us  that  Q  is  Schur-convex.  We  need  to  examine 
two  cases,  maXjjr’j}  =  7i,  and  the  alternative.  Assuming  the  former,  straightforward  algebra  shows 
that  the  difference  is  bounded  from  below  by 

-i-  1)  -  </7(i,r,)]  -  ^  [<t>{i,r2)  -  </>(*, >"2  -  1)]  + 

1=1  t=i 

(<■/>( ?’i  +  l,''i  +  1)  -  -  (r,  -  r2)(</>(r2  +  l,rz)  -  <j>{r2,r2  -  1)). 

Both  of  the  two  summations  above  are  positive,  because  ^(i,  r)  increases  in  r.  Since  </>(i,  r)  is  convex 
in  r  and  r\  >  rz,  it  also  follows  that  <A(7,ri  +  I)  —  >  (f>{i,r2)  -  <i>{i,r2  —  1)  for  every  i.  Thus 

the  positive  summation  above  dominates  the  negative  summation,  and  the  desired  inequality  will 
hold  if 

+  l,ri  -b  1)  -  </»(r2,r2))  -  (r,  -  r2){<i>{r2  +  l,r2)  -  (t>{r2,r2  -  1))  >  0. 

Since  </>(r,  r)  is  a  convex  function  of  r,  we  have 

ri+l-r2 

(l>(r]  +  +  [)  -  (tjirz^rz)  -  ^2  (</>(r2  +  1,^2  +  i)  -  <?!>(r2 -f  t  -  l,r2  +  i  -  1)) 

1=1 
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ri-f  l-rj 

^  +  1,  ^2  +  1)  -  </>(r2,  r2)) 

i=l 

-  (n  -  T2){(t>{r2  +  1, r2  +  1)  -  <f>{T2,ri)). 

From  this  inequality  we  see  that  the  desired  bound  will  hold  if  {(j>{r2  +  l,r2  +  1)  -  ^*2))  > 

{(j>iT2  +  —  (I>{t'2,t'2  —  1))  ■  The  convexity  of  s  miplies  that 

</>(r2  +  1,7-2  +  1)  -  Hr-i.r-i)  =  E[s{\  +  -  s(l  + 

>  E[s(X(r2)-.s(X(r2-l))] 

=  «j!>(r2  +  1 ,  r2)  -  <;6(r2 ,  r2  -  1 ), 


as  needed. 

The  argument  for  the  case  when  t\  ^  maxj{rj}  is  almost  exactly  the  same,  and  so  is  omitted. 
The  Schur- convexity  of  Q  gives  us  a  stochastic  majorization  for  GS  synchronization. 

Proposition  4.2  Let  .s  be  an  inavasing  convex  function  with  s(0)  =  0,  suppose  the  space  cost 
of  holding  k  IVf/s  in  one  proce:ssoi''s  queue  for  one  time  unit  is  s(k),  and  suppose  the  branching 
distribution  is  ILR.  Define 


0{r\,...,rK)  =  »'A.)  +  (m^{rj }  -rfc)<?i(rfe+  l,r*:)|  , 

fc=i  \i=i  ^  ) 

to  measure  the  space- time  cost  with  respect  to  s  of  executing  some  generation  q  under  GS  synchro¬ 
nization,  where  the  each  pwcessor  i  has  ri  generation  q  WUs.  Then  Q  is  Schur-convex  on  iV^,  so 
that  whenever  m  -<  m' , 

•  E[C;{Wfim))]  <  E[cJiW„{m))]  for  q  =  0,1 . 

•  The  expected  total  space-time  cost  using  GS  synchronization  is  no  worse  under  m  than  under 
m  : 

00  00 

El'£,e(W,(m))]  <  £;Ee(lC,(m))l  whenever  the  expectation  exists. 

q=0  g=0 

4.3  Reliability 

Yet  another  application  of  majorization  is  to  the  question  of  whether  the  hardware  will  successfully 
execute  the  entire  computation.  We  suppose  that  the  computation  “fails”  if  any  processor  having 
a  non-empty  queue  fails.  Observe  that  this  definition  permits  the  computation  to  successfully 
complete  even  if  a  processor  dies  before  the  entire  computation  is  finished,  provided  the  failing 


1.5 


l)ro(essor  is  itsolf  already  liiiislipd.  We  will  show  that  if  the  branching  distribution  is  ILR  and  a 
])rocessor's  time-tu-railiire  distribution  has  an  increasing  hazard  rate  function,  the  the  probability  of 
failure  under  m  is  no  greater  than  that  under  m',  whenever  m  -<  m'.  Conversely,  if  the  branching 
distribution  is  ILK  and  tlu'  processor  failure  distribution  has  a  decreasing  hazard  rate  function,  then 
the  relial)ility  under  m'  is  better  than  that  under  m.  The  result  is  proven  for  TS  synchronization. 

Suppose  that  iirocessor  Ts  time  to  failure  is  the  random  variable  Z,,  with  an  monotone  hazard 
rate  function  \(  u).  It  is  well  known  that 

r*r{Z  >  t}  =  exp{—  f  X{u)  ds}. 

Jo 

If  A(  It)  is  nondecreasing  in  u,  then  —  A(  u)  du  is  concave  in  t,  which  is  to  say  that  log  Pr{Z  >  t} 
is  concave.  Conversely,  if  A(u)  is  decreasing,  then  logPrjZ  >  t}  is  convex. 

It  follows  (.'LE.l  in  [10])  that  when  A(u)  increases,  the  product 

K 

Pr{Z,  >  <.}  (3) 

i=I 

is  Schur-concave,  or  ecpiivalently,  that  -7v(ti , . . is  Schur-convex.  When  A(u)  decreases  then 
Tl{t\ _ ,//y)  is  Schur-convex. 

If  processor  i  is  assigned  iin  Wlls  initially,  it  ends  up  processing  WUs  total.  This  is  also 
|)rocessor  fs  |)rocessing  time  under  the  assumptions  of  S(»F  scheduling,  TS  synchronization,  and 
unit  execution  cost  |)er  WII.  (liven  =  (*,.  for  i  =  1, . . . ,  A',  equation  (3)  gives  the  probability 

that  every  i)rocessor  executes  all  WUs  without  processor  failure.  The  unconditional  probability  is 
obtained  by  taking  the  expectation  with  respect  to  the  joint  distribution  of  S//(m): 

Prjevery  processor  executes  all  its  WUs  before  failing)  =  E['R,{Sti(m))]. 

Lemma  2.2  asserts  that  .Vyv  is  ILRC  if  the  branching  distribution  is  ILR.  It  follows  from  Theorem  3.1 
that  when  A(n)  is  increasing,  A[7v(.SV{m))]  is  a  Schur-concave  function  of  m.  This  proves  the 
following  |)roposition. 

Proposition  4.3  Supposi  tin  hazard  rate  Jxinction  A(ti)  for  the  time,  to  pioces.'ior  failwr  is  incrcas- 
iiuj,  and  suppose  tin  hranrhiiifi  distribution  is  ILR.  Let  '){m)  be  the  probability  that  every  processor 
executes  all  its  WUs  witlunit  processor  failun..  Then  under  TS  synchronization  and  SGF  scheduling, 
whenever  m  -<  m'  we  have  7(771)  >  The  inequality  is  ixversed  if  A(u)  is  decreasing. 

5  Assignment  of  Processor  Pools 

Our  last  application  of  stochastic  majorization  concerns  a  problem  where  a  large  number  P  of 
processors  are  to  be  partitiemed  among  a  smaller  number  T  of  complex  tasks.  Parallel  processing 
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can  be  applied  to  the  tasks  to  accelerate  execution  time.  We  assume  that  a  task  requires  that  all 
of  its  generation  i  WUs  to  be  executed  before  any  of  its  generation  i  +  1  WUs  are,  but  that  all 
generation  i  WUs  may  be  processed  in  parallel.  As  before,  the  overall  system  may  use  either  TS 
or  GS  synchronization. 

Let  g(X,  711)  give  the  time  required  by  7n  processors  to  execute  X  WUs.  We  assume  that  g{X ,  771) 
is  convex  in  m,  e.g.,  g{X,7n)  —  X/m,  and  that  g(0,  m)  =  0. 

Suppose  there  are  K  initial  WUs.  We  may  describe  our  assignment  of  71  processors  to  these 
WUs  with  vector  m,  whose  component  gives  the  number  of  processors  assigned  to  the  WU. 
.41so  let  i  denote  the  random  number  of  WUs  associated  with  generation  q  of  task  i.  Under  GS 
synchronization,  the  time  required  to  complete  the  generation  is 

7,(m)  =  max{g(A,,i,mi),(?(Ag,2,m2),...,fif(A,,A-,mA-)}- 

Under  our  assumptions,  £[7,(711)]  is  a  symmetric  convex  function  of  m  (B.4  Proposition  in  [10]), 
showing  that  £[7,(771)]  <  £[7,(771')]  whenever  m  ^  771'.  It  follows  immediately  that  the  overall 
expected  finishing  time  under  GS  synchronization  is  no  worse  under  m  than  under  m' . 

Under  TS  synchronization  the  finishing  time  is 

00  00 

p{m)  =  max{5^i((A,,i,77ii),...,5^p(A,,A-,7nA,.)}. 

q=0  q=0 

A  sum  of  convex  functions  remains  convex,  whence  E[p{7rn)\  is  symmetric  and  convex  in  m.  When 
771  ^  771'  we  are  assured  that  the  expected  finishing  time  under  TS  synchronization  is  no  worse 
using  771  than  it  is  with  m' . 

6  Conclusions 

This  paper  explores  the  application  of  majorization  to  the  problem  of  assigning  a  large  number  of 
stochastically  complex  (but  probabilistically  identical)  tasks  onto  a  multiprocessor.  Using  a  model 
of  workload  based  on  branching  processes,  the  theory  we  develop  establishes  a  partial  ordering 
among  possible  assignment  of  tasks  to  processors.  We  show  that  the  quality  of  an  initial  assignment 
persists  through  stochastic  transformations  of  the  workload,  and  that  the  ordering  can  be  taken 
with  respect  to  a  wide  range  of  objective  functions  including  those  measuring  finishing  time,  space- 
usage,  and  reliability.  We  also  show  how  the  theory  applies  to  the  processor  partitioning  problem. 
The  utility  of  the  theory  lies  in  the  generality  of  the  objective  functions  that  can  be  considered,  and 
in  the  fact  that  optimal  solutions  can  be  identified  even  when  constraints  are  placed  on  potential 
assignments. 
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A  Appendix 


In  this  appendix  we  prove  some  claims  made  earlier  in  the  paper. 

The  ILRC;  condition  upon  which  our  -<scx  results  depend  involves  the  notion  of  totally  positive 
functions.  Chapter  LS  of  [10]  is  the  source  for  the  following  definition. 

Definition  A.l  (Totally  Positive  Function)  Let  A  and  B  be  subsets  of  the  real  line.  A  function 
a  :  A  X  B  -*  R  is  said  to  be  totally  positive  of  order  k,  denoted  TPk,  if  for  all  m,  1  <  m  <  k  and 
all  X]  <  X2  <  . . .  <  x,n,  y\  <  y-2  <  -  .■<  yin  (xi  e  A,  yj  e  B) 


n{xuy\) 


Olixuym) 


det 


>  0. 


We  will  use  the  following  result  (l8.A.4.ain  [10]). 

Lemma  A.l  If  K  is  T Pm  and  L  is  TP^,  and  a  is  a  cr-finite  measure,  then  the  convolution 

M{x,y)  =  j  K{x,z)L(z,y)d(T{z) 

The  relationship  between  total  positivity  and  ILRC  distributions  is  direct.  Given  any  integer¬ 
valued  nonnegative  probability  mass  function  /  we  may  define  the  function  o/  :  iV  x  iV  — ♦  [0, 1]: 

o/(z,i)=  f^'\x). 

n f  is  T  Pi  iff 

for  all  i  <  j,  m  <  n.  But  this  is  equivalent  to  saying  that  f^^\  i.e.,  that  /  is  ILRC. 

The  reason  for  our  interest  in  ILRC  distributions  /  is  that  their  convolution  functions  a/  satisfy 
three  criteria  required  by  Theorem  3.J.2  of  [10] 

•  aj{x,y)  —  0  whenever  y  <  0; 

•  o/  is  totally  positive  of  order  2; 

•  nf{x  +  z,y)  =  /  0/(3:,  u)o(2,  y  -  n)di;(u),  for  some  measure  w  on  TV. 
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Tliporein  conclusion  is  that  if  m  =  , th/c)  €  iV^,  /i  is  counting  measure,  and 

0  :  — ►  is  Schur-convex,  then 

■y(m)=  (4) 

. vk)  ,=i 

is  Schur-convex  on  .  Theorem  3.1  is  a  restatement  of  this  result,  where  Vu,dv(u)  =  1;  because 
(flit',  Vi)  is  a  probability,  we  recognize  that  7((m))  expresses  the  expected  value  of  <^(y). 

-<ca.s  Results 

We  next  consider  the  -<oas  ordering.  In  this  case,  we  are  able  to  obtain  the  analogue  of  Theorem  3.2, 
save  that  the  result  holds  for  com])letely  general  branching  distributions.  We  first  must 

introduce  a  little  more  terminology,  and  develop  an  intermediate  result. 

A  random  ’’^rtor  X  =  {Xi,  .  .,X,^)  is  said  to  have  exchangeable  components  if  the  joint  dis¬ 
tribution  of  A'l, . . .,  Xn  is  invariant  under  permutations  of  its  components.  Our  basic  <c(is  results 
rest  on  the  following  observation. 

Lemma  A. 2  Let  X ,Y  be  nonnegative  random  tiariables,  and  Z  =  {Z-[,Z-2)  be  a  random  vector 
with  nonnegative  exchangeable  cojuponcnts.  Assutne  that  X,  Y  and  Z  are  independent  r.v.’s  and 
define  U  =  (.Y,  T)  and  V  =  {X  +  T,0).  Then 

Z  U  -<,cas  Z  V , 


Proof.  Let  (p  ■  R+  — "  ^  convex  symmetric  function.  Define  the  function  ij’  :  >  R 

as  rj>{a)  =  E[<j){Z  -|-  o)],  Va  G  Rl^.  Since  Z  has  exchangeable  components,  V’  is  also  a  convex 
symmetric  function. 

Now  U  ■<  V  a.s.  from  which  it  follows 

HU)<iiV)  =>  E[x(iU)]  <  E[xl’{V)], 

=>  E[<f>{Z  +  U)]<  E[(I>{Z  +  V)], 

=>  Z  +  U  <,„,Z  +  V. 

■ 

The  result  extends  easily  to  R’^. 
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Lemma  A. 3  Let  X ,Y  be  any  nonne.gative.  random  variables,  let  Z  =  (Z] ,  Z2,  •  •  • ,  Z„)  G  be  a 
random  veetor  loith  independent  components  such  that  Zi  and  Z-z  have  the  same  distribution.  As¬ 
sume  that  X ,  Y ,  and  the  components  of  Z  aiT  mutually  independent  and  define  U  =  (X,  0,  •  •  -  ,0) 

and  V  =  (X  +  Then 

Z  U  -Kcas  Z  V . 


Proof:  Let  (j) :  St"'  ^  ^  be  a  symmetric  convex  function.  Now,  <!>  is  symmetric  and  convex  in  the 
first  two  arguments.  Therefore,  we  can  condition  the  values  of  Zj,  3  <  y  <  K  to  be  Zj  and  apply 
the  previous  lemma  to  obtain 

Ex.Y.Zx.Zi[<l>{U)\Z:i  =  23,  ■■■,Zk  =  -A']  <  Ex,Y,Zi,Z2[<l>{V)\Z-.i  =  23,  •  •  • ,  ^A'  =  ^A'] 

Removal  of  the  conditioning  on  3  <  i  <  K  yields  the  desired  result.  ■ 

We  are  now  prepared  to  prove  Theorem  3.3.  Let  m'  be  any  mapping  vector  where  there  are 
processors  i,  j  such  that  m',  >  in'-.  Without  loss  of  generality  we  may  take  i  =  1  and  j  =  2,  and 
let  m"  be  the  mapping  vector  obtained  from  m'  by  moving  one  WU  from  processor  1  to  processor 
2.  We  will  apply  lemma  A. 2.  Interpret  Z\,Z2  as  convolutions  of  initial  WU  rewards,  X 

as  the  convolution  of  -  m'z  —  1  initial  WU  rewards,  T  as  a  single  initial  WU  reward,  and  each 
Zj  for  j  >  2  as  the  convolution  of  m'j  initial  WU  workloads.  The  application  of  lemma  A. 3  yields 
R{m.")  R{m'). 

The  incremental  movement  of  a  task  from  a  heavily  loaded  processor  to  a  more  lightly  loaded 
processor  corresponds  to  the  more  general  notion  of  a  “transfer”  [10].  It  is  known  that  whenever 
X  ^  y,  then  x  can  be  constructed  from  y  with  a  finite  number  of  transfers,  where  each  transformed 
vector  is  always  dominated  under  by  its  predecessor.  Consequently  if  m'  is  a  mapping  vector 
with  m  ^  m',  then  one  demonstrates  that  W{m)  -(cas  W{m')  through  a  repeated  application  of 
Lemma  A. 3  to  the  secpience  of  transfers  that  transmute  m'  into  m.  This  proves  the  result. 
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